Approximately Minwise Independence with Twisted Tabulation

A random hash function h is ε-minwise if for any set S, |S| “ n, and element x P S, Prrhpxq “ minhpSqs “ p1 ̆ εq{n. Minwise hash functions with low bias ε have widespread applications within similarity estimation. Hashing from a universe rus, the twisted tabulation hashing of Pǎtraşcu and Thorup [SODA’13] makes c “ Op1q lookups in tables of size u1{c. Twisted tabulation was invented to get good concentration for hashing based sampling. Here we show that twisted tabulation yields Õp1{u1{cq-minwise hashing. In the classic independence paradigm of Wegman and Carter [FOCS’79] Õp1{u1{cq-minwise hashing requires Ωplog uq-independence [Indyk SODA’99]. Pǎtraşcu and Thorup [STOC’11] had shown that simple tabulation, using same space and lookups yields Õp1{n1{cq-minwise independence, which is good for large sets, but useless for small sets. Our analysis uses some of the same methods, but is much cleaner bypassing a complicated induction argument.